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The 1972 assessment activities in the Massachusetts 
schools concentrated on the areas of citizenship and science^ 
utilizing two seventh grade samples of 10,000 students each, with 
each group completing only one assessment form. Th^ testing program 
utilized National Assessment Items for age 13, as well as a few items 
from the Measurement Research Center to replace items not released by 
National Assessment for the objective being measured. The following 
procedures were carried out: (1) two four-page pamphlets, one for 
Citizenship and one for Science, were prepared listing the objectives 
used by National Assessment in these curricula area; (2) a rating 
scale to indicate priority of objectives was prepared and sent to 
each of the 57 schools used in the sample population; rating 
categories applied by each school to. each objective were: (a) 
objective with highest priority; (b) objective of importance 
requiring extensive evaluation; (c) objective important, but not 
requiring extensive evaluation; (d) objective not of importance; 
omission will have no effect; and (e) objective not applicable to 
this school system. Ob jective results of the assessment were not 
available for inclusion. Attachment A to the report provides samples 
of the Citizenship and Science Objectives, and Attachment B is a 
summary of the ratings of all Science Objectives and identifies the 
number of items to be used in measuring each objective, (DB) 
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I ntroductlon 

Massachusetts, a nowccmer to state planned testing and evaluation activities, . 
Is operating undar an assessment design or plan based upon the following 
hlerarchafl classification of testing conditions: 

K Administration of tests to large groups for normative Interpretation 

2. Interpretation of achievement In light of aptitude or ability 

3. Testing for mastery or criterion reference testing 

4. Individualized tracking and evaluation of pupil achievement 

As the Commonwealth sets out to move ultimately toward a highly Individualized 
testing philosophy, minimizing as much as possible over Interpretation of 
comparative or normative statistics, It has skipped stage Number 1 above, 
moved last year through a demonstration of achievement-aptitude comparisons, 
and Is now demonstrating classification Number 3, mastery testing * 

Testing during the 1970-71 school year involved administration of the California 
Test of Academic Aptitude and the Comprehensive Test of Basic Skills to 90,000 
fourth grade pupils. All data reports emanating from the Division of Research 
and Development related achievement {''obtained") scores to aptitude (''anticipated 
scores. Stanlne scales for all subtests and total scores based on the 
"difference" scor'Ss provided an opportunity for both Individual school and 
school system interpretation. 

Philosoph y underl ying cu rrent experimentation 

Because of the recognition of differences In pupil populations, differences 
In curricula content, and difforences In educational philosophies among school 
systems In Massachusetts, selection of school test content by the State Depart- 
ment of Education for administration to all schools was open to serious 
question. Questions have been raised regarding the audacity of the Department 
to select for £»dmInistratlon last year to ail schools even a test of the basic 
skills. 

Rocoqnizlng the above condition, the following policies were adopted by the 
Department's Task Force planning the. current year's assessment activities: 

K Measurement would relate to the content areas of citizenship and 
science and would reflect levels of mastery implying criterion 
reference rather than norm reference testing. 
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2. Performance would be reported In terms of Individual Item mastery 
omitting total scores. 

3» Teachers In the schools participating In the sample populations 
would have a voice In deciding objectives to be measured. 

Sampling procedures 

Two mutually exclusive samples were randomly selected for tho seventh grade 
testing^ so that no group of students would be asked to complete more than 
one assessment form. Schools were grouped according to the size of their 
seventh grade enrollment* Each group was designated as a stratum and five 
strata were Identified* From each of the five strata, representing small 
to large enroMments, ^ proportional nuJnber of schools was randomly selected. 
Eoch ten percent sample was composed of fifty-seven schools representing 
about 10^600 pupils. 

Order! nci Objectives 

The availability of National Assessment objectives In citizenship and science 
and the jyotentlal availability of Items measuring these objectives, thankfully 
discouraged Msssachusetts from reinventing the wheel — a characteristic of 
education In the 70»s. This Is not to Imply that the wheel does not need 
retreading, but the prepared objectives of National Assessment at least rnoved 
the Ck^nmonwealth away from the starting gate and Into the first turn. 

Mora specifically, and with the assistance of Westlnghouse Learning and t4RC^ 
Its subsidiary, the following procedures were carried out. 
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Two four page pamphlets, one for Citizenship and one for Science 
were prepared listing the objectlvea used by National Assessment 
In these curricula areas of citizenship and science, 
(See Attachment "A"), 
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A rating scale to Indicate priority of objectives was* prepared 
and sent to each school used In the population of 10,000 In each 
Seventh grade sample. 



Rating categories applied by each school to each objective ore 
obbrevlatod below: 



(1) 
(2) 
(3) 
(4) 
(5) 



Objective with highest priority. 

Objective of Importance requiring extensive evaluation. 
Objective Important, but not requiring extensive evaluation. 
Objective not of Importance; omission will hove no effect* 
Objective not oppttcable to this school syst«m» 
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Presented below Is an Illustration of the ranKlng or reordering of two 
objectives In the area of Citizenship. Items ranked 1 were glvon a 
priority value of Items rankad 2 v/ors given a priority valua of 2; 
Items ranked 3 wore given a priority value of 1; Items rankad 4 were 
given a priority value of -1; and Itoms rcjnkad 5 were glvon a priority 
valua of -2, . 

A. Help maintain law and order by under- 1 = 73 x 3 » 219 

standing th^* need for law and ort^v. 2 = 44 x 2 « 88 

3 15 X 1 » 15 

4 « 2 x-1 = -2 
5=3 x-2 » ^ 

OiTitts = 0 314 



B. Help maintain law and order by being 1 = 61 x 5 - 183 

conscious of right and wrong behavior; 2 = 51 x 2 » 102 

encourage ethical and lawful behavor 3 *» 22 x 1 = 22 

In others. t i """l " "1 

5 « 2 x-2 =» 

C-nlts = 0 302 



Items A and B above Indicate a high priority having a total rating value 
of 314 and 302 respectively. RanKlng of Items ranged from e low of 54 
for the objective "Help maintain law and order by protesting unjust rules 
openly" to Item A above having a value of 314. 

P I ann I ng jtie Tests 

Since the 7th grade was the level chosen for the sample, National Assessment 
Items for age 13 were applicable to this group. In certain instances, 
however. National Assessment had not released Items pertaining to a selected 
objective. It was necessary, therefore, for Measurement Research Center to 
select from Its own files or develop a few Items having construct validity 
for the objective being moasared. 

Attachment B presents a suin.'nary of the ratings of all Science Objectives and 

Identifies the number of Items to be used messurlng each objective. 

A suggested cut-off point for the cumulative rating was decldod at a leva 

of 112. Cumulative ratings below this point ref loctsd objocttves not Included 

In the test. 

It can be recognlzod that tho ejbovo process attornptsfi to obtain Input at th© 
local level assuring a higher degres of currlcular validity thon could be 
expected from the administration of any already publUhod Citizenship and 
Science tests. 

Analysis of Results 

At the time of presentation of this paper. It was not posslblo to present 
objective results. A report Indicating level of msstery In terms of each 
ftemand objective will bo prepared for each school district. Also provided 
wilt be an item and objectives surrmary or analysis Indicating mastery date 
for th9 total statewide sample in both Citizenship and Sclonco testa* 
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Limitations of fho 5tuc 3y 

As a result of procjross to doto, cfirtain limitations are aUroady notad 
and futuro limitations or conslderatloni^ wit! bo oxpGcted* . Throa of 
thesa aro Identified In the following paragraphs: 

K CltUenshfp Itoms .appear to be moro subjoctlvo than Sclenco 
Items. Thli; may bo rsftected In ths higher priority ratings 
givon Citl2on$hip Itoms as comparod to Science. Strong 
. object lOHD to soma of the objectives were forthcoming from 
toechers. 

2» SoiT;o teachers requested recolpt of tha Items for acceptance 
or rojoctlon rather than the objectives* These roquesis 
v/ero not granted as thoro was feoljng that thoro would 
bu direct teaching of the Item rather than the objective.. 
The concern merits further study however. 

3. Because of the acceptance of a great number of objectives, 
relatively few Items appeared In each test with reforonce 
to th^r objective limiting conclusions regarding mastory. 

Further Study 

An attempt will be made to ascertain the reaction of teachers, administrators, 
and In cortelh Instances, School Board personnel, regarding both the 
measurement technique. end the mastery levels of pupils. The effect of the; 
testing on curriculum development will noted during the coming academic 
year. 
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At-iQChmont "A" 



niustrations of the Citizynship Objectives appsar belo/f*. 



in. HELP f-V\IfJTAIiI LAW AND ORDER 



A. Understand tiie nestl for law and order. 



3. Arc conscious of ritjnt and v;ronn bahayiof, encourage ethical 
and lawful behavior In others. 

_C. Comply with public law and school rules, 

p. Help authorities in specific cases. 

E. Protest unjust rules openly. 



F. Inform themselves about the law. 



Each of the above objectives were reacted to at o priority rating consistent 
with the priority definitions. 



Illustrations as follows are samples of the Science Objectives: 

11. POSSESS THE ABILITIES AND SKILLS NEEDED TO ENGAGE IN THE PROCESSES 
OF SCIENCE 



A. Ability to identify and define a scientific problem. (This 

. includes recognizing and defining the problem present in a given 
situation* and Isolating the problem from extraneous material.) 

B. Ability to suggest or recognize a scientific hypothesis. (This 
includes syntiiesliing data pertinent to a problem, and recognizing 
the possibility of testing a hypothesis.) 

C. Ability to propose or select validating procedures. (This 
includes planning or recognizing m adequate plan for the 
collection of data» making logical predictions, and designing 
apparatus sat-ups.) 

_E, Ability to interpret data; i.e., to ccinprehend the meaning of 
data and to. recognize, formulate, and evaluate conclusions and • 
generalizations on the basis of information known or given. 



It is important in considering the illustrations obova to place rmjor 
emphasis on the process involved without too detailed an 9valiiat1cn of 
the definitivaness of the objective statejnsnts. 



Summary ratings of Science objectives with suggested number of items. 
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